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(57) Abstract 

The present invention relates generally to design and evaluation of research and development, technology transfer, and learning- 
by-doing, and more particularly to the determination of production plans and the prediction of innovation. A preferred embodiment 
comprises a method for determining a production plan comprising the steps of. defining a plurality of production recipes such 
that each of said production recipes is a vector of n operations; selecting a cunrent one of the production recipes; evaluating 
the current production recipe to determine its cost; modifying the current production recipe to create a trial production recipe 
(130); evaluating the trial production redpe to determine its cost (140, 150); and assigning the trial production recipe to the 
current production recipe if the cost of the trial production recipe is less than the cost of the current production recipe. 

(57) Abr6g6 

Cette invention, qui a trait, d'une manidre gen^rale, d la conception et d revaluation dans les domaines de ia recherche- 
d^veloppement, du transfert de technologies et de I'apprentissage par la pratique, conceme. plus pr6cis6ment la determination 
de plan de production et la prediction dinnovation. Un mode de realisation prefer^ porte sur una methode de determination de 
plan de production consistant ^ detinir plusieurs procedures de production de sorte que chaque procedure de production solt un 
vecteur de n operations, ^ choisir une des procedures en cours. ^ I'evaluer afin de determiner son coOt. d modifier la procedure 
de production en cours afin de creer une procedure de production d'essai (130), S evaluer cette derniere afin de determiner son 
cout (140, 150) et a affecter cette procedure de production d'essai d la procedure de production en cours si son cout est 
infeneur S celui de la procedure en cours. 
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(54) TiUe: A SYSTEM AND METHOD FOR DETCRMINING PRODUCTION PLANS AND FOR PREDICTING INNOVATION 
(57) Abstract 



The present invention relates generally to design and evaluation of 
research and development, technology transfer, and leaming-by-doing, and 
more particularly to the determination of production plans and the prediction 
of innovation. A preferred embodiment comprises a method for determining 
a production plan comprising the steps of: defining a plurality of production 
recipes such tliat each of said production recipes is a vector of n operations; 
selecting a current one of the production recipes; evaluating die current 
production recipe to determine its cost; modifying the cuirenl production 
recipe to create a trial production recipe (1 30); evaluating the trial production 
recipe to determine its cost (140, 150); and assigning the trial production 
recipe to the current production recipe if the cost of die trial production 
recipe is less than the cost of the current production recipe. 
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A SYSTEM AND METHOD FOR DETERMINING PRODUCTION PLANS AND 
FOR PREDICTINd INNOVATION 

FIELD OF THE INVENTION 

5 The present invention relates generaliy to design and evaluation of research and 

developmenU technology transfer, and leaming-by-doing, and more particularly to the 
determination of production plans and the prediction of innovation. 

BACKGROUND OF THE INVENTION 
According to the neoclassical model a production plan is merely a point in input- 

10 output space. The neoclassical model has been extended to accommodate inter-tcmporal 
features such as the variability over time of factory supplies, uncertainty about the production 
process, and uncertainty about prices. The neoclassical model of production is not, however, 
fully dynamic since it does not provide a microeconomic basis for explaining technological 
evolution due» for example, to leaming-by-doing, education and training, research and 

15 development, or technology transfer. 

In this regard, macroeconomics is ahead of its microeconomic foundations. Li his 
celebrated article on leaming-by-doing. Arrow (1962) (K.J. Arrow, The economic 
implications of leaming-by-doing, Review of Economic Studies^ 29:155-73, 1962) accounted 
for the observed fact that unit production costs could fall even in the absence of capital 

20 accumulation and R&D effort. Arrow attributed the increased productivity to leaming-by- 
doing on the shop floor by production workers and managers. Arrow modeled Icaming-by- 
doing as a positive macrocconomic production externality: increases in "manufacturing 
experience** — as measured (for example) by cumulative gross investment — lead to increased 
productivity. Several other macro models of technological progress were based on some 

25 production externality. See, for example, Clemhout and Wan (1965) (S. Clemhout and H. 
Wan Jr., Leaming-by-doing and infant industry protection, Review of Economic Studies, 
32:233-240, 1965), Shell (1967) (K. Shell, A model of inventive activity and capital 
accumulation, in K. Shell, editor. Essays on the Theory of Optimal Economic Growth y pages 
67-85, 1967), Stokey (1988) (N.L. Stokey, Learning by doing and the introduction of new 

30 goods. Journal of Political Economy, 96(4):701-719, 1988); Romer (1990) (P.M. Romer, 
Endogenous technological change. Journal of Political Economy, 98(5):57l-5102, 1990); 
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and Lucas (1993) (R-E, Lucas Jr., Making a miracle, Econometricay 61(2):251-272, March 
1993). 

Another class of (not unrelated) macro models of technological evolution was based 
on non-conventionai factors of production. Uzawa (1965) (H. Uzawa, Optimum technical 

5 change in an aggregative model of economic growth. International Economic Review, 1965) 
introduced in a simple growth model "human capital, " the stock of which can be increased 
by devoting resources to education. In the hands of Lucas (1988) {R.E. Lucas Jr., On the 
mechanics of economic development. Journal of Monetary Economics^ ll'.y-Aly 1988); 
Caballe and Santos (1993) (J. Caballe and M. Santos, On endogenous growth with physical 

10 and human capital, JPE^ 101 (6): 1042-1067, December 1993) and others, this human capital 
model (with clearly modeled externalities) became a staple for analyzing productivity growth. 
Shell (1966) (K. Shell. Toward a theory of inventive activity and capital accumulation, 
American Economic Review, 56{2):62-68, May 1966); Shell (1967); Shell (1 973) (K. Shell, 
Inventive Activity, industrial organization and economic growth, in J.A. Mirrlees and N.H. 

15 Stem, editors. Models of Economic Growth, pages 77-100. 1973); Romcr (1986) (P. M. 
Romer, Increasing returns and long-run growth. Journal of Political Economy, 94:1002- 
1037, 1986); and Romw (1990) introduced into growth models the macro variable 
^'technological knowledge" The Shcll-Romer model combines "technological knowledge" 
(or "stock of patents**) with production externalities and increasing returns- to-scale to analyze 

20 the role of industrial organization in growth, the dependence of growth on initial conditions, 
and other important macro problems. 

Macrocconomic models based on production externalities and/or non-conventional 
inputs have been useful in raising important issues about public policy toward technology and 
in explaining observed increases in aggregate output, but the inadequacy of the 

25 micToeconomic foundations of these models is a serious problem for the analysis of 
production. 

The neoclassical economic model of production (see e.g., Debreu (1959, Chapter 3) 
(G. Debreu, Theory of Value: An Axiomatic Analysis of Economic Equilibrium, 1 959), 
Arrow and Hahn (1971, Chapter 3) (K.J. Arrow and F. Hahn, General Competitive Analysis, 
30 1971), and the references to the literature therein) is a reduced-form model of existing tech- 
nological possibilities. Each firm is endowed with a technology set — a set of technologically 
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feasible input-output combinations. These technology sets are assumed to be fixed parame- 
ters of the neoclassical economy. 

A simple example with one input and one output is described in FIG. 16. A 
neoclassical production plan is a point (x, y) where x a 0 is the quantity of the input and 

5 where y a 0 is the quantity of the output. The (shaded) set 7 is the set of all feasible 
neoclassical production plans. The northwest boundary of T is the production possibility 
frontier (PPF). Points on the PPF represent the "efficient" production plans according to the 
neoclassical model, since no other plan can be found that gives either more output for the 
same input or less input for the same ou^ut In this example, the PPF is linear, i.e., 

10 production exhibits constant returns to scale. The production function is 

y=^jc. 

where the positive scalar 0 is the slope of the PPF. 

From the viewpoint of neoclassical production models, production plan A in FIG. 16 

is "efficient." Production plan B is "inefficient." In fact, A strictly "dominates" B: pair A 
15 yields more output with less input than pair B. ^ and S are nearby (or similar) production 

plans. C is distant from A and B. 

Now look at FIG. 1 6 firom the engineering point of view. Suppose that engineers tell 

us that there are only two knov^ processes (Recipe 1 and Recipe 2) for producing this output. 

Recipe I supports all pairs (:c, y)iO that satisfy^ i Ojc, where 6 is a positive scalar. Recipe 2 
20 supports all pairs (x, y) i 0 that satisfy^ s 0'jc, where 6' < 6 is a positive scalar. The 

production pairs AdndC lie on the ray>' = 6x, while pair B lies on the rayy = 0'x (indicated 

in FIG. 16 by the dashed line). 

Suppose Recipe 1 is from an engineering viewpoint very different firom Recipe 2. 

even though pair A is close to pair B -- would it ever be rational to produce at B? The answer 
25 is yes. Suppose that Recipe 2 is relatively untried. Using Recipe 2 might lead to the 

discovery of recipes close to 2, but with lower production costs than Recipe I (and Recipe 2). 

A case could be made in some circumstances for using only Recipe 2 and in other 

circumstances for using both Recipe 1 and Recipe 2 simultaneously. 

According to the neoclassical model, production pairs A and B arc close (or similar), 
30 while the production pairs ^ and C are far ^art (or dissimilar). But as measured by their 

recipes, A and 5 are by assumption far apart (or dissimilar) if Recipe 2 is used for B, while 
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plans A and C must be based on precisely the same recipe. Recipe 1. Furthermore, the 
neoclassical model does not accurately represent the opportunities facing the production 
manager. He must jointly choose the recipe and the production plan. Having chosen 
Recipe l» only a production pair from the PPF should be chosen. Having chosen Recipe 2, 
5 only production pairs from the dashed ray in FIG. 16 should be chosen. All other pairs in T 
(i.e., those not satisfying >» = 0x or y = 6'jc) result from waste of some output or some input 
when using one or both of the two basic engineering processes. 



SUMMARY OF THE INVENTION 

10 A preferred embodiment comprises a method for determining a production plan 

comprising the steps of: defining a plurality of production recipes such that each of said 
production recipes is a vector of n operations; selecting a current one of said production 
recipes; evaluating said cunrenl production recipe to determine its cost; modifying said 
cuiient production recipe to create a trial production recipe; evaluating said trial production 
15 recipe to determine its cost; and assigning said trial production recipe to said current 
production recipe if said cost of said trial production recipe is less than said cost of said 
current production recipe. 

A second preferred embodiment comprises a method for predicting technological 
innovation comprising the steps of: 
20 defining a model comprising: 

a plurality of production recipes such that each of said production recipes is a 
vector of n operations; and 

a plurality of model parameters; and 
executing said model comprising the steps of: 
25 selecting a current production recipe; 

evaluating said current production recipe to detemune its cost; 
modifying said current production recipe to create a trail production recipe; 

and 

assigning said trial production recipe to said current production recipe if said 
30 cost of said trial production recipe is less than said cost of said current production recipe. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The following detailed description, given by way of example, will best be understood 
in conjunction with the accompanying drawings in which: 

FIG, 1 is a flowchart diagram illustrating the general steps of a preferred embodiment. 
5 FIG. 2 illustrates an analytical representation of a production recipes model used in a 

preferred embodiment 

15 FIG. 3 illustrates the 5-neighborhoods used in a preferred embodiment. 

FIG. 4 illustrates a technological gr^h. 

FIG. 5 illustrates the analytical representation of cost reduction dynamics used in a 
10 preferred embodiment. 

HG. 6 is a table (Tabic 1) of estimated progress ratios for various industries. 
FIG. 7 is a table (Table 2) summarizing the parameters used in a preferred 
embodiment 

25 PIG, 8 is a table (Table 3) summarizing the results of a 20-run per parameter vector 

1 5 experiment run on a focused parameter set. 

FIG. 9 is a table (Table 4) summarizing the results of a 20-run per parameter vector 
experiment on a random parameter set. 

FIG. 10 is a three part table (Table 5) illustraUng the prediction of the sample mean of 
the estimated progress ratios by OLS scoring from the focused parameter set. 
20 FIG. 1 1 is a three part table (Table 6) illustrating the prediction of the sample mean of 

35 ti^g estimated progress ratios by OLS scoring from the random parameter set. 

HG. 12 is a three part table (Table 7) illustrating the prediction of the sample 
standard deviation of the estimated progress ratios by OLS scoring from the focused 

^ parameter set. 

25 HG. 13 is a three part table (Table 8) illustrating the prediction of the sample 

standard deviation of the estimated progress ratios by OLS scoring from the random 
parameter set 

^ FIG. 14 is a three part table (Table 9) Ulustrating the prediction of the mean 

improvement percentage per measured batch by OLS scoring from-the random parameter set 
30 HG. 15 is a three part table (Table 10) illustrating the prediction of the mean 

curvature mis-specification by OLS scoring from the random parameter set 
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FIG. 16 illustrates weaknesses of neoclassical production theory. 
FIG. 17 is a histogram of estimated finn progress ratios. 
FIG. 18 illustrates the phenomenon of plateauing. 
FIG. 19 illustrates the SFS (Slow Fast Slow) Mis-specification. 
5 FIG. 20 is a graph of a base parameter experience curve, after one production run. 

FIG. 21 is a graph of a smooth landscape, after one production run, 
FIG. 22 is a graph of a rugged landscape, after one production run. ' 
FIG. 23 is a graph of a landscape with « = 10, e = 1. and j = 100, after one production 

run. 

1 0 FIG. 24 is a graph of the base parameter experience curve, with the same parameters 

as in the gr^h in FIG. 20. but averaged over twenty production runs. 

FIG. 25 is a graph of a smooth landscape, with the same parameters as in the graph in 
FIG. 21, but averaged over twenty production runs. 

FIG. 26 is a graph of a rugged landscape, with the same parameters as in the graph in 
15 FIG. 22. but averaged over twenty production runs. 

FIG. 27 is a graph of a rugged landscape, with the same parameters as in the graphs in 
FIGS. 22 and 26, but averaged over fifty production runs. 

FIG. 28 is a graph of a landscape with the same parameters as in the graph in FIG. 
23, but averaged over twenty production runs. 
20 FIG. 29 illustrates a way in which parameters are chosen in the preferred 

embodiment. 

FIG. 30 illustrates effects of varying «. the number of operations in the recipe, on the 
sample mean of the estimated progress ratios (p) and the sample standard deviation of the 
estimated progress ratios (s^,). 
25 FIG. 31 illustrates effects of varying s, the number of possible states for each 

operation in the recipe, on the sample mean of the estimated progress ratios (p) and the 
sample standard deviation of the estimated progress ratios (j^). 

FIG. 32 illustrates effects of varying e, the number of other operations that affect the 
cost of each operation, on the sample mean of the estimated progress ratios (^) and the 
30 sample standard deviation of the estimated progress ratios (5^) (in the case s = 2). 
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FIG. 33 illustrates effects of varying the number of other operations that affect the 
cost of each operation, on the sample mean of the estimated progress ratios (p) and the 
sample standard deviation of the estimated progress ratios {s^) (in the case j = 10). 

FIG. 34 illustrates effects of varying S, the maximum number of operations changed 
5 per trial, on the sample mean of the estimated progress ratios ( p ) and the sample standard 
deviation of the estimated progress ratios (s^). 

FIG. 35 illustrates effects of varying the number of trials, on the sample mean of 
the estimated progress ratios (p) and the sample standard deviation of the estimated progress 
ratios (jp). 

10 FIG. 36 illustrates plateauing effects of varying n, the number of operations, on the 

sample mean of the estimated progress ratios ( p ) and the standard deviation in the percent of 
trials that result in improved productivity {s.). 

FIG. 37 illustrates plateauing effects of varying 5, the number of possible states for 
each operation in the recipe, on the mean percentage of trials that result in improved 

15 productivity ( z ). the standard deviation in the percent of trials that result in improved 
productivity (jj, and other parameters. 

FIG. 38 illustrates plateauing effects of varying e, the number of other operations 
that affect the cost of each operation, on the mean percentage of trials that result in improved 
productivity { F) and the standard deviation in the percent of trials thai result in improved 

20 productivity 

FIG. 39 illusu^tes plateauing effects of varying 3, the maximum number of 
operations changed per trial, on the mean percentage of trials that result in improved 
productivity ( 2 ) and the standard deviation in the percent of trials that resuU in improved 
productivity (.y J. 

25 FIG. 40 discloses a representative computer system 401 0 in conjunction with which 

the embodiments of the present invention may be implemented. 

DETAILED DF^CRIFTION OF PREFERRED EMBODIMENT 
A preferred embodiment is a system and method for modeling technological 
30 innovation. One of the more important aspects of a preferred embodiment is in the 
description of the production plan. To the usual input-output specification is added a 
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description of the underlying engineering recipe employed. Describing how one recipe is 
related to another then allows one to build models that suggest which types of technologies 
are likely to be uncovered in the course of ordinary shop-floor operations (Icaming-by-doing), 
which R&D programs are most likely to be successful, and which types of technologies are 

5 ripe for transfix from one firm (or economy) to another. 

In a preferred embodiment, a production recipe is described by a vector of basic 
production operations such as heating, mixing, stirring, shying, boxing, internal factory . 
transportation, and so forth. For given outputs, the input requirements for each of the 
operations depends on the instruction (or setting) given for that operation and the instructions 

10 given for some of the other operations. Hence the embodiment allows for production 
externalities within the firm. 

A specific application of the (more general) production recipes approach (along with 
the nascent technology approach) of the preferred embodiment is (he construction of a model 
of shop-floor Icaming-by-doing. It is assumed that the firm employs a single input to produce 

15 a single output and that, for a given fixed recipe, this process entails constant returns to scale. 
It is also assumed that the fimi*s output stream is prcdetenmined. The method allows for 
deviations from the currently reigning technology, but assumes that such production trials (or 
"errors**) are not directly controlled by the finn. It is assumed that a newly discovered recipe 
is either accepted or rejected merely on the basis of current cost efficiency relative to that of 

20 the reigning technology. 

By correctly choosing the basic parameters of the model, one is able to match the 
basic statistics and important qualitative phenomena from observed experience 
curves — including the mean progress ratios (an inverse measure of the slope of the 
experience curve) and their standard deviations, plateauing (runs without improvements), 

25 cuTvatiu^ bias, and sensitivity to the length of the production run. The disclosed method of 
modeling of recipes works for one input, one output, constant rctiuns cases for which the 
focus is either the short term or the intermediate term. 

A preferred embodiment comprises the general steps of applying the machinery of the 
production recipes model to a specific problem (in a preferred embodiment, the leaming-by- 

30 doing problem), refining the parameters of the model to fit the details of the probletn, and 
then utilizing the model to predict which nascent technologies are the most promising. The 
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subject disclosure describes a system and method for constructing a microeconomic model of 
technological evolution. To the existing (or "currently, available^') technologies of the 
neoclassical production model, nascent technologies arc added, which include both 
undiscovered technologies and forgotten technologies. One might be skeptical about any 
5 modeling of undiscovered technologies. While existing technologies can be verified by 
current engineering practice, undiscovered technologies cannot On the other hand, 
^5 practicing production engineers and business managers are not reluctant to base important . 

business decisions on forecasts of technological progress in the firm's manufacturing 
operations. In fact, one of the most reliable analytic tools in production management is the 
10 engineering experience curve (or "learning" curve), which projects existing unit production 
costs for a given product into its future unit production costs. Among production engineers, 
marketing managers, business executives, and even corporate directors, empirical learning 
curves are far better known and more firequently used than are empirical production functions 
25 or empirical cost functions. 

1 5 A production recipe w is a complete list of engineering instructions for producing 

given outputs from given inputs. In the preferred embodiment it is assumed that the firm uses 
a single input to produce a single output. It is also assumed that, given the recipe choice, 
^ there is no waste in production. It is assumed that in production run f, the firm produces y, i 

0 units (production engineers often use a precise (but different for each output class) unit of 
20 measurement called the "batch") of output by employing > 0 units of input (hereafter 
"labor") based on the recipe <*>,. It is also assumed that there is no uncertainty about the 
production process. Ut - y/n, be unit labor cost. Then - (OV. ^.)^ where, for fixed u,, e 
is the average cost fimction. Iff first falls and then rises as^ is mcreased, then the average 
cost curve is U-shaped. If C is independent of^,, then there are constant returns to scale. 
25 The method of representing technologies utilized in a preferred embodiment is an 

improvement on the approach taken in several methods of modeling technological innovation 
in which there are two types of technologies, advanced and backward (and two types of firms, 
^ also advanced and backward) (see, e.g.. Shell (1973) and Grossman and Helpman (1991) 

(G.M. Gixjssman and E. Helpman, hmovation and Growth in tiie Global Economy, 1991)). 
30 Advanced firms have access to both the advanced and backward technologies, but backward 
firms axe resUictcd to the backward technology. Let w. be the advanced recipe and a)„ be the 
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backward recipe. In this literature, the strong non-crossing (in the absence of non-crossing, 
there might not be a most advanced (or a most backward) recipe) assumption is made, so 

0<{0';o)J< f(y;t^iJ< «'foreachy>0. 
For the advanced firm, the set of recipes is given by 
5 Cl.«{w„<»>b}. 
For the backward firm, the set of recipes, Qb, is given by 

In the subject specification, the above very simple model is generalized to allow the 
firm to choose not merely fit)m (at most) two recipes, but instead fi*om a large general set of 

10 recipes, Q. The forma! model in what follows is restricted to constant returns to scale, so that 
the unit labor requirement depends only on the recipe employed. The labor requirement for a 
given recipe is not typically known with certainty. Instead there is associated with each 
recipe o> a probability measure over the set of labor requirements. Although it is also 
assumed that there is a (relatively small) subset of currently available recipes Q^i <= Q, and 

15 that the respective labor requirements for each of these recipes is known with certainty, this 
assumption is not critical. 

In the preferred embodiment, a production recipe is described by a vector of basic 
production operations such as heating, mixing, stirring, shaping, boxing, internal factory 
transportation, and so forth. For given outputs, the input requirements for each of the • 

20 operations depends on the instruction (or setting) given for that operation and the instructions 
given for some of the other operations. Hence one allows for production externalities within 
the firm. These intra-firm production exleroalities are crucial to the analysis. 

The method of a preferred embodiment is to use the production recipes model to 
provide a microeconomic model for observed leaming-by-doing in production, and then use 

25 computer simulations to discover which nascent technologies are the most promising. A 
complete model of production is specified (the IRF) (which includes inputs, outputs, and 
recipes; as well as current technologies and nascent technologies) and then "closed" so as to 
model shop floor productivity improvements and hence to explain the observed empirical 
features ofthefirm*s experience curve. There are three reasons for doing this: (l)The 

30 leaming-by-doing model is important in its own right for economic theory and economic 
policy. It would be worthwhile to understand the micro sources of the productivity increases 
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and what promotes them, rather than merely representing this process as a fixed 
macroeconomic externality. (2) Empirical experience-curve analysis is central to 
management science and management practice. It would be desirable if these experience 
curves could be explained in terms of basic micro economics. (3) The method of production 
5 recipes (and nascent technologies) is quite general, with possible applications for modeling 
Ri&D, basic research, and technology transfer. In the disclosed preferred embodiment, the 
general method is illustrated by a concrete application. There are three reasons why leaming- 
by-doing is a good candidate for the preferred embodiment: (1 ) Empirical studies of 
engineering experience curves are abundant. (2) Leaming-by-doing permits one to be 

10 relatively less sophisticated in modeling the purposiveness of the economic agents, so that 
one can focus — for the time being — on the relatively sophisticated model of technology. (3) 
The one input/one output leaming-by-doing model allows one to use — ^with only minor 
modifications — a tested computer simulation program. 

In the production recipes/nascent technology approach of a preferred embodiment, the 

15 construction of a model of shop- floor leaming-by-doing to determine promising nascent 
technologies, it is assumed that the firm employs a single input to produce a single output and 
that, for a given fixed recipe, this process entails constant returns to scale. It is also assumed 
that the firm's output stream is predetermined. The method allows for deviations from the 
currently reigning technology, but assumes that such production trials (or "errors'*) arc not 

20 directly controlled by the firm. It is assumed that a newly discovered recipe is either accepted 
or rejected merely on the basis of current cost efficiency relative to that of the reigning 
technology. 

These strong assumptions allow the employment of a variant of KaufTman's NK 
model (see Kauffinan and Levin (1987) (S. Kauffman and S, Levin, Toward a general theory 

25 of ad^tive walks on rugged landscapes. Journal of Theoretical Biology^ 1987) and 

Kauffinan (1988, 1993) (S. Kauffinan, The evolution of economic webs, in P.W. Anderson, 
KJ. Arrow, and D. Pines, editors. The Economy as an Evolving Complex System, 1988; S. 
Kauffinan, Origins of Order: Self-Organization and Selection in Evolution, 1993)) to 
analyze the dynamics of manufecturing costs. The NK model was originally designed for an- 

30 alyzing asexual biological evolution. In the evolutionary biology interpretation, it is assumed 
that the **fitness" of a creature can be represented by a scalar. The corresponding assumptions 
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for leaming-by-doing are the single output, the single input, and constant returns to scale, 
which together allow the scalar "fitness" to be replaced by the scalar "current technological 
efSci«ic/* (the inverse of current unit production cost). In the biological interpretation, it is 
assumed that genetic changes occur at random and that fitter creatures immediately replace 
5 those that are less fit. In the present interpretation, the corresponding assimiptions are that 
shop- floor trials take place at random and that the reigning recipe is replaced by the new 
recipe if and only if the new recipe is more efTicient in the short nin - i.e., recipe selection is. 
myopic. 

The preferred embodiment is not restricted to myopic recipe selection. It would 
10 certainly be routine for one skilled in the art to extend the model of leaming-by-doing to 
allow for both foresight in selection of the reigning recipe(s) and some control by the fimi of 
the rate and direction of experimentation. For the latter, there must be costs of 
experimenting. These would include the output losses from pilot-project retrogressions, the 
opportunity costs of sampling other recipes, and additional resource costs of experimenting 
15 with distant recipes. 

A preferred embodiment works for one input, one output, constant return cases for 
which the focus is either the short term or the intermediate term. 

FIG. 1 is a flow diagram illustrating the method of a preferred embodiment. At step 
100 a set of one or more production recipes, each recipe comprised of n operations, is 
20 represented as a set of recipe vectors {w} in R". At step 110 a technological graph F is 

constructed whose nodes are the recipe vectors w. At step 120 a measure of distance between 
the recipe vectors is defined. At step 130 a production recipe trial on T is defined to be a 
recipe vector within a distance be {1,2, n} of a currently adopted recipe vector. At step 
140 productivity gain or loss is defined in terms of an estimated progress ratio, and at step 
25 150 production recipes are evaluated by measuring the productivity gain or loss of trials on F. 
in a preferred embodiment, production is assumed to involve n distinct engineering 
operations. The recipe w 210 can then be represented by 

o) = {ci)V.., <i)' wOeia" 

where w' represents the instructions for operation i for i = 1, n. See FIG. 2. It is assumed 
30 that for each operation / the set of possible instructions is discrete. These choices may be 
"qualitative" (e.g., whether to use a conveyor belt or a fork-lift truck for internal transport) or 
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they may be "quantitative" (e.g., the setting of the temperature or other knob on a machine). 
In the latter case, the variable being adjusted is approximated by discrete settings (think of the 
knob "clicking" from one setting to another). In particular, it is assumed that is an integer 
which satisfies 
5 o>'e {l,....s} 

for i = 1, . . . . where s 220 is a positive integer. Hence the number of recipes (we shall 
sometimes refer to the recipe vectors and the recipes themselves interchangeably, since 
context will clarify whether actual recipes or vectors in R" are being referred to) is finite and 
given by 

10 #Q = ^. 

The finiteness of the set Q - {recipe vectors <*)} has serious consequences. A model 
based on a finite space of recipes docs not permit long-run productivity growth. The model 
with a finite set of recipes is, however, quite appropriate for modeling the intermediate term 
productivity improvements observed in the manufacture of specific goods (such as a 

1 5 particular model of a microprocessor) over their relatively short product lives (measured in 
months, years or — at the very most — decades). 

It is assumed that the unit labor cost of operation /, ({)*((»>), is a random variable whose 
distribution function is defined on Consider two distinct recipes, u and<o'. The random 
variables 4»'((i)) and 4>'(coO are not necessarily independent In fact, (J)' depends on the 

20 instructions, co', for operation / and possibly on (some of) the instructions for the other 
operations, o)*'. (With minor abuse of notation, one could then have denoted the unit labor 
costs of operation / by 4>(«*>^ ^'0* or more simply, (J>'(o>).) The labor requirements are 
assumed to be additive; hence 

ji 

25 (t>(w) = J] <|>'(o>) 

where 4>(^) is the unit cost of 

production employing recipe w. For w fixed, (t)((i)) is a random variable. If o) is allowed to 
vary over Q, then 4»{o)) is a random field, A random field is a slight generalization of a 
stochastic process to allow the argument (in this case w) to be a vector (as opposed to being a 
30 scalar such as ''time"). For the special case in which w - 1 , <t>(6>) is then an ordinary 
stochastic process. Denote by P'(o)) the realization of the random variable <!>*(«»>)• The 



13 



wo 00/20983 



PCT/US99/22911 



realization of the random variable {j>(<i>) is ?(a>) = {'((o). If w varies over Q, the family of 

(•I 

realizations C(a)) is called the landscape (of Ihe random field <)'(<*>))* 

There are six basic parameters in the disclosed model of shop-floor leaniing-by-doing: 
(1 ) the number of production operations, (2) j, the nimiber of possible instructions or 
5 settings for each operation^ (3) e, ihe externality parameter that gives the number of 

operations whose settings affect the costs of one operation, (4) 6, the maximum step size per 
trial, (5) t, the number of trials made on the shop floor per measured batch, and (6) T, the 
length of the production nm. 

The above discussion shows how the neoclassical notion of technological distance can 

1 0 be misleading. One needs a measure of distance that captures the similarity or dissimilarities 
of the inherent production processes rather than the relative efficiencies of production pairs or 
their relative scales. "Distance" between recipes will, of course, depend on the application. 
If one takes a shop-floor perspective — as is done in the leaming-by-doing embodiment of 
the method — then <»> is near 'w if these recipes arc the same except for, say, one temperature 

1 5 setting. If moving from a> to o)' represents the substitution of fluorine for chlorine where 
fluorine was not formerly in use, then one would probably think of w and o>' as very far apart 
in the shop-floor metric. But for an R&D problem, the appropriate metric might be altogether 
different. In the chemistry research lab, for example, the distance would in this case be small, 
since every chemist is aware that the elements chlorine and fluorine are likely to react 

20 similarly because they arc from the same column of the Periodic Table of the Elements. 

The preferred embodiment assumes that the set Q can be described so that distances 
are meaningful from the appropriate technological perspective. A formal defmition follows. 

Definition (Distance): The distance d (o>.g>') between tiie recipes o> and (»>' is the 
minimum number of operations which must be changed in ordw to convert o) to co'. Since 

25 changing operations is symmetric, ((0,(0 ') = rf {(*>', Ci)). Example: Assume that w and a>' 
differ only in the /th component. Then d (w.co') = 1 when to' = 1 and (o>0' = 2 or when <o^ = 1 
and (CO*) = 37. 

This definition of distance makes the most sense when the instructions are merely 
qualitative. If instead the instructions can be represented by ordinal settings (such as 
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temperature), then the distance notion should be different. If the instructions for operation / 
in the above example had been ten^>erature settings, then the recipe with its fth entry equal to 
2 would be closer to co than the recipe with its /th entry equal to 37. In particular, 2**C is 
closer to 1 than is 37*C. If settings are ordered, then a wise strategy for the firm might be 
5 to change if possible the setting in the same direction that led to the most recent 

improvement. If 2°C is an improvement over 1 ^'C, perhaps the next trial should be 3**C. 
Introduction of ordinal settings and more complicated distance measures is also within the 
claimed scope of the preferred embodiment In general, the requirement is that the set Q have 
some reasonable notion of distance imposed on it. 
1 0 Definition (Neighbors): Let N^iij)) be the set of /-neighbors of recipe <i>: 

where i is a positive integer. 

Let y?6(co) = {(!)' 6 Ci| w'6 u^,_ft Ntiw)} 

be the set of recipes at least distance one from (o but not more than distance 6 e { 1, .... n) 
15 fromo). See FIG. 3. 310 \s 2i d-neighborhood d 2tO. 

With the above definition of distance between recipes, it is straightforward to 
construct the technological graph T (see FIG. 4). The nodes or vertices 410 of F are the 
recipes o) e fl. The edges 420 of F connect a given recipe to recipes distance 1 away (i.e., to 
the elements of iV|{<o)), 

20 Next comes the definition of the input requirement field, which is a full description of 

all the basic technological possibilities facing the firm. 

Definition (IRF): The input requirement field (IRF) is the random field (t>(w) (defined 
over the vertices of the technological graph V). 

In order to obtain concrete results, one must further specify the IRF. In the next 

25 description, the relationship between the random variables <|){(.)) and <|)((i>'), which in general 
are not independent, is specified. Ai^er that is specified the fiinctional forms for the (|)'(t»)). 
Except for some of the underlying stochastic structure, this specifies the IRF, but it does not 
"close the model.** To do that, one needs a complete algorithm of which recipes are chosen 
for production, what is learned by the firm fipom its experience, and how much of this is 

30 remembered. 



15 



wo 00/20983 



PCT/US99/22911 



10 



15 



20 



25 



It is assumed that the costs of a given operation depend on the chosen setting for that 
operation and possibly on the settings for some (but nol necessarily all) of the other 
operations. Define the connectivity indicator efj by 

, J 1 if the choice of setting for operation / affects 
ej* - I the labor requirement for operation J 

[ 0 otherwise 

for /,y = 1 n. Since the choice of the setting for the ith operation always affects the costs 
5 for the /th op^tion, 

for / = 1 n. The number of operations with costs affected by operation i is given by 

e' - t e- 

r\ 

10 for I = 1,..., while the number c, of 

operations that affect the costs of operation / is given by 

for / - I,..., /I. Define the set , the 
30 15 set of operations cost-relevant to operation /, by 

£-{/-6{K....«}k,= l} 

fori= 1, n. 

The simplifying assumption is made that each operation is cost-affected by {eA) other 
operations, so that 
20 #£, = e, = e 

for f = 1 , w, where ee {!,.„,«}. Under this assumption^ the labor requirement of any 
40 given operation is affected by the settings for that operation and the settings for exactly (e-i) 

other operations. Therefore, there are exactly f permutations of the settings that affect the 
costs of operation L Each of these is a sub-recipe cost-relevant to operation i. Let {/,,..., i^) 
IS denote the elements in ttie set i?, = {j je', = 1 } for i = 1,..., /». Then denote by (u",..., a 
sub-recipe cost-relevant to operation i (where for convenience is defined to be equal to 0 
for /= n (due to typographic difficulties, the symbol il, e.g., should be identified 
throughout with the symbol /;). The set of sub-recipes cost-relevant to operation / is a 
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projection of the y recipes into J* sub-recipes. There are ns* such sub-recipes in all. The 
stochastic unit labor requirement for operation ; based on the sub-recipe (o)'',.„w*) can be 
written as 

<J)V'»"., or (frVO 
5 for I = 1 /*t with only slight abuse of notation. 

The parameter e plays a crucial role in the preferred embodiment. If e = I, there are 
no (intra-firm) external effects among the operations. Each of the operations could as well 
have taken place in separate firms, since in this case there can be no gains from coordination. 
With e - 1, one would also expect the two random variables <|){g)) and <t>(c»>') to be highly 
10 correlated if t*)' is close to (o, since by definition co' and o> would have many instructions in 
common and hence <|>'(ti)) 4)'(w') for most /. The larger the parameter e, the less correlation 
one would be expect between <t)(a>') and (J)(w) even for w' close to o). This is because the 
change in the instructions for one operation affects the costs of several other operations. 
Hence e is an inverse measure of the correlation between (J)(a)') and (|){G)) for d)' close to o). 
15 The corresponding landscape l(a>) (a realization of the IRF) is typically "correlated," or 
"smooth,'* for small values of while f(o)) is typically "uncorrclated," or "rugged," for large 
values ofe. 

This concludes the disclosure of the general production recipes model; next the 
specification illustrates the preferred embodiment by applying the general model to the 
20 problem of modeling leaming-by-doing. 

U is assumed that shop floor workers follow the recipe {or blueprint) provided by 
management, but from time to lime, they make small modifications in the current recipe. 
These modifications are referred to as **trials" in the subject specification. Depending on the 
context or the mterpretation, the trials may also be thought of as "errors" or "informal 
25 experiments.'* 

A trial occurs when (1) at least one operation / in the production recipe o> is modified 
and (2) that modification is observed and is evaluated by the fum (perhaps by the quality- 
control engineers). It is assumed that modifications occur during the production of a batch, 
and that observation and evaluation occur when the production of the batch is completed. It 
30 is further assumed that there is only one trial per "quality-control batch." This assimiption is 
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not restrictive: since the quality-control batch size, may or may not equal the measured 
hatch size, B , used in data-gathering, the batch deflator, t, is introduced, t satisfies 

T = B/5 

where B is defined so that exactly one trial is made during the production of the quality- 
5 control batch. Hence if t is a positive integer it is interpreted as the number of trials per 
measiired batch. In general, t > 0 is interpreted as the average number of trials per measured 
batch. Note that t k 1 if B ^B, and T < 1 if B < J?. 

One would expect T to depend on the particular type of manufactured good. The trials 
parameter t is likely to be higher for airframes than for computer chips, because the number 

10 of operations, n. is likely to be higher in the airframe case. The trials parameter, t, also 
depends on management practices, corporate culture, and worker psychology. In a tight 
production enviromnent, there would be fewer defects, but fewer trials. In a looser 
production environment, there would be more defects, but also more trials. Hence 
management is likely to be keen to attempt to control x if possible to achieve an optimal 

15 balance between "sloppiness" and "inventiveness." 

A trial can be interpreted in at least three (non-exclusive) ways. The first 
interpretation is that the trial is a small scale experiment in production to which the firm does 
not fiilly commit. This would be a model of R&D in which the only cost of the R&D activity 
is the missed opportunity for investigating alternative recipes during the period in question. 

20 A natural alternative is to assume that the firm must commit to the new production 

recipe in order to sample it: If recipe is chosen during the production run /, then the labor 
requirement for run / will be the realization of 4>(ci),)- ^ Ibis case, unit costs may actually 
increase from one period to the next, i.e., cost retrogressions could occur. 

A third interpretation is possible. One can think of the firm operating a large number 

25 of distinct production subunits in parallel. Each sub-unit begins a production run at time t 
with an assmned labor requirement i)» the unit cost for run / - 1 . During the course of 
production run ^ a trial in one production sub-unit leads to production by its subunit using 
recipe w*. The associated labor requirement for this production sub-unit is then fi(a)'), the 
realization of 4>(o>'). However, because there are many production sub-units, the average per- 

30 batch cost of production is close to i), the unit cost of the (pre-trial) reigning recipe. 
Consequently, in this scenario, one can think of firms trying new recipes, without substantial 
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15 



sacrifices in current labor requirements. This third interpretation is utilized in the preferred 
embodiment, beciause it allows simulations to be based on an existing, tested computer 
progriam. 

iO The next description is of the trial dynamics and the memory of the firm. 

5 Production trials occur on the shop floor level at the rate of 1 per quality control batch 

B (or T per measured batch B). It is assumed that the trial recipe is at distance of at least one 
but no greater than 6 e { 1 , 2...., n) from the currently adopted recipe (o. It is assumed that 
the probability of a trial is unifonn over the neighborhood A^6(c*>). (See FIG. 3 for an 
illustration of a neighborhood Nt,{<j>) where g) = (U3). j = 4, n = 2, and 6 = 2.) The 
1 0 probability distribution of trials could be different. For example, a scheme that loads more 
20 probabihty on the recipes closer to ^^>^,^ would in general be more realistic. Such a 

modification would be a matter of weighting the probability distribution as function of the 
distance from ci),.i, would be obvious to one of ordinary skill in the art. , The quantitative 
effect of this change would be to reduce the big-step effect for any given 6. but the qualitative 
1 5 effects would not be altered. Tliat is, the extent of modification of existing recipes is limited 
to recipes that differ from the currently prevailing recipe by no more than t6 operations — the 
number of trials per production run multiplied by the number of operations of the recipe that 
30 can be altered in a single trial. 

It is assumed that the recipe adoption process is very simple. The firm is myopic: if 
20 o),.| is the prevailing recipe and o)',., is the trial recipe, then the prevailing technology for 
period /, (d, , will be given by 

35 
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0 for w'^y^^Cto,.,) 



The system (1) defines the cost-reduction dynamics (see FIG. 5). This is a process in 
30 which the firm moves from vertex to vertex of the technological graph T that underlies the 
IRF. If 6 = 1, then the firm moves along the edges of the graph F to the next vertex. 
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For the nascent sub-recipes, it is assumed that the random variables <J>'(a)") are 
independent and identically distributed (i.i.d.) and uniform on [0.1/n]. Hence if all the sub- 
recipes of 0) are nascent, then the support of the random variable <t)(ca) is contained in 
[0. 1]. The density function of 4)(co) is then n-sh^ed and symmetric about V^, the modal 
5 labor requirement. The density is 0 for the extreme labor requirements, 0 and 1; otherwise it 
is positive. If u*' is currently available, then 4>'(o)*0 is a degenerate random variable, in which 
all of the probability is massed on a scalar ^ (w) € [0. 1/n]. If the recipe to is currently 
available, then <J)(o)) is a degenerate random variable, in which all of the probability is massed 
on a scalar J{o)) e[0,l]. If m of the sub-recipes of the recipe co arc nascent (while the 

10 remaining n - m sub-recipes are currently available), then the support of <|)(to) is [?, C + (w//i)], 
where f is a scalar in [0. (n - m)in\ Probability (n - m)M is massed on On the interval (C. { 
+ {jtUn)\ probability density function (p.d.f ) is o-shaped and symmetric Hm>\. (If ot = 
2, then the p.d.f on (f, e + (m/n) is a **tent.") 

The assumption that the support of <K<«>) is contained in [0,1] is not innocuous. The 

15 fact that a zero labor requirement is possible could in principle be disturbing, but it is 
assumed that the probability of such a free lunch is zero. The assumed boundedness of unit 
cost is more serious. In actuality, many untried recipes turn out to be totally useless - i.e., 
widi the realization ?(co) = +«. The assumption ofboundedness from above of unit labor 
requirements is an optimistic assumption about prior beliefs on untried recipes. 

20 A formal description of experience curves and a review of the (largely empirical) 

literature on them is now given. The statistical outputs of the existing studies were used as 
inputs for the simulations of the disclosed embodiment. The existing experience curve 
analyses are not redone; instead the calculated regression coefficients and their standard 
errors arc used as observations, 

25 Few empirical phenomena in production economics or management science are as 

well documented as the experience curve. It depicts the decline in the finn's per-unit labor 
requirements (or costs) with the cinnulative production of a given manufactured good Fol- 
lowing Wright's (1936) (T. P. Wright, Factors affecting the cost of airplanes. Journal of the 
Aeronautical Sciences, 2: 122-128, 1936) study of the anfeme industry, this pattern has been 

30 investigated for many different goods and industries. 

The usual parametric form of the experience curve is the power curve. 
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*, = «Y,^, (2) 

1-1 

where ^, = f(a),) is the unit labor requirement for production run r, y,., = , is the 
cumulative output up to (but not including) run r, i» > 0 is the learning coefficient, and 

5 fl > 0 is the labor needed to produce the first batch of the good. In the case of airframes, one 
airframe is equal to one measured batch. In this case, /, would be the serial number of the 
last airframe in production run t. The learning coefficient represents the rate at whidi the 
productivity increases as the firm acquires "experience." 

There is an impressive amount of empirical work on the engineering experience 
10 curve. Wright's law is that unit cost is related to , „ cumulated output, by the power 
function i, = fl( J',./)"^, so that the progress ratio p is given by /» = 2 = 0,79. If one allows 
for a more general power law, one gets that 

1 5 where b>0 and p = 2**. The progress ratio /? is a decreasing function of the exponent 6, the 
learning coefficient. Post-Wright empirical smdies suggest that observed progress ratios live 
m the range of about 60-95% with a mode of about 8 1-82%. The existing empirical literature 
reports not only mean progress ratios but also their standard deviations. 

A commonly used measure of productivity improvement is the progress ratio. The 

20 progress ratio p is related to the learning coefficient b by 

;? = 2-* or/> = -log207). 
The percentage cost reduction when cumulative output is doubled is 100(1 - p). Ifb = , 
then p = 0.79 ("Wright's Law*')* so the percentage cost reduction is 21 %, If = 0.30, then p = 
0.81 (sec die mode in FIG. 17). A small progress ratio is an indicator of rapid cost 

25 improvement, while a higher progress ratio indicates less cost improvement. For example, a 
p value of 0.85 (an "85% curve") means that per-imit cost falls by 1 5 percent when 
cumulative output is doubled,/? - 0.60 means per-unit cost falls by 40% when cumulative 
output is doubled, p = 1 means that unit cost is constant, and p > 1 means that unit cost is 
increasing. 

30 The power law given in Equation (2) yields the straight line in log/log space 
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log«, = logfl- /^logY,.,. 

where is the average unit labor requirement (or cost) of producing the goods with serial 
numbers greater than Y,,2^^ ^ess than or equal to Y, . (. 
5 In Table 1 (FIG. 6) are displayed some estimates of the learning coefficients and the 

progress ratios from firm-level experience curves (by industry). From this and the review 
articles by Conley (1970) (P. Conlcy, Experience curves as a planning tool. IEEE Spectrum^ 
7:63-68, 1970), Yelle (1979) (L.E. Yelle. The learning curve: Historical review and 
comprehensive survey, Decision Sciences^ 10, January 1979), Dutton and Thomas (1984) 

10 (J.M. Dutton and A. Thomas, Treating progress functions as a managerial opportunity. 

Academy of Management Review, 9(2):235-247. 1984). Muth (1986) (J. Muth, Search theory 
and the manufacturing progress function. Management Science^ 32:948-962, 1986), and 
Aigotc and Epple (1990) (L. Argote and D. Epple. Learning curves in manufacturing. 
Science, 247:920-924, 1990). one concludes that the salient characteristics of experience 

15 curves are: (1) The distribution of progress ratios ranges from 55% (rapid progress) to 105% 
(slow — indeed negative! — progress) and "centers" on about 81-82%. See Table 1 (FIG. 6) 
and FIG. 17. (2) Distinct production processes and goods are associated with their own 
ranges of values for the estimated progress ratio p. See Table I (FIG, 6). (3) There is 
variation in progress ratios not only among firms in the same industry, but also among 

20 different plants operated by a single firm. See Alchian(1963) (A. Alchian, Reliability of 
progress curves in airfi^jne production, Econometrica, 31 :679- 693, 1963), Dutton and 
Thomas (1984. pp.236-239). and Epple, Argot and Devadas (1991) (D. Epple, L. Argote, and 
R. Devadas. Organizational learning curves: A method for investigating intra-plant transfer 
of knowledge acquired through leaming-by-doing. Organization Science, 2(1): 58-70, 

25 February 1991). (4) The specification of the OLS (ordinary least-squares) statistical model, 
= aV***,. ,e, with log - N{fi, o), is imperfect in (at least) two ways: (A)There are ^^plateau 
effects^ in the observed data (see Abemathy and Wayne (1 974) (W.J. Abemathy and JC 
Wayne, Limits of the learning curve. Harvard Business Review, September-October 1974)): 
(i) Improvements occur after relatively long stretches of constant labor requirements, and (ii) 

30 improvements in labor productivity cease beyond some (sufficiently large) cumulative output 
(see Conway and Schultz (1959) (R.W. Conway and A. Schultz Jr., The manufacturing 
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progress funcUon, Journal of Industnal Engineering, 10:39-54. 1959) and Baloff(1971)(N, 
Baloff, Extension of the learning curve - some empirical results. Operational Research 
Quarterly, 22:229-^40, 1971). The (hypothetical) empirical experience curve of FIG. 18 
illustrates both types of platcauing. The labor requirement { is falling from f = 0 to / = 10, but 

5 not strictly monotonically. There is an interior plateau at batches 3-5. Productivity 

improvements cease after batch 7, providing a terminal plateau. (B) There is curvature mis- 
specification (see Levhari and Sheshinski (1973) (D. Levhari and E. Sheshinski. Experience 
and productivity in the Israeli diamond industry, Econometrica, 41 :239-253, 1973) and Epple 
et al. (1991, pp. 65-69)). Instead of a straight line in log/log space, the data suggest that an S- 

1 0 shaped curve would often (but not always) fit better: often the data suggest concavity of the 
ftmction over the early batches, but convexity of the function over the later batches. This 
could be called the SFS phenomenon. Cost improvement is first slower, then/aster, and 
finally slower than suggested by the straight-line log^og fit. See FIG, 19. (hi many observed 
production runs and many experiments, the right-hand tail of the productivity plot is truncated 

1 5 before the suggested convex range of the function can be observed.) (5) Industry experiaice 
curves (in which the data on cost as a function of cumulative output is averaged over several 
firms) are smoother than the corresponding single-firm experience curves, which in turn are 
smoother than single-plant experience curves. . There are fewer plateaus and the lengths of the 
interior plateaus are shorter for the averaged data. See the survey by Button and Thomas 

20 (1984). 

An analysis of the comparative dynamics of the disclosed model of shop floor 
leaming-by-doing is given next. In particular, the effects of varying the following basic 
parameters of the model are described: the number of operations n \ the number of 
instructions per operation s\ the externality parameter e; the maximum number of steps per 

25 trial 5; the number of trials per batch x; and the length of the production run T, on the two 
basic predictions of the model: the sample mean p of the estimated progress ratios and the 
sample standard deviation of the estimated progress ratios, and on two measures of model 
mis-specification: plateauing (or its inverse, the improvement percentage z); and curvature 
(or SFS) in log/log space. 

30 The basic parameters of the model are summarized in Table 2 (FIG. 7). The third 

column gives the range over which each parameter is defined. Non-integer values oft would 
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have been possible. The interpretation of (say) x = is that a trial occurs once in every three 
production runs. 

Computations were performed on a Dell Dimension XPS Pro 200 PC with a Pentium 
Pro 200 MHz processor and Windows 95 operating system (version 4.00.950 B). The core 

5 program used was written by Bennett Levitan, building on work by William Macready and 
Terry Jones. Regressions were performed, summary statistics were computed, and plots were 
generated with SPSS Windows version 7.5 and Matlab Windows version 4.0. The routines . 
used to compute and generate the simulations from the random parameter set were written by 
Auerswald. Both Levitan's and Auerswald's programs incorporate a random number 

10 generator written by Terry Jones, based on the algorithm of Knuth (1981, pp. 171-72). 

The most fundamental unit of analysis is a single realization of the production run, 
examples of which are displayed in FIGS. 20-23. A "point" in one of these figures gives the 
normalized log of the unit labor requirement for the currently prevailing technology. The 
horizontal axis is the log of time (or, equivalently, the log of the accumulated number of 

1 5 quality-control batches to date) . A production run is then a *Svalk" on a landscape (i.e., a 
realization ?(a)) of the IRF 4)((o)). The line in one of these figures is the OLS fit to the points 
in the figure. 

There are many approaches to defining in detail the e "extcmalit/' coimections from 
one operation in a given recipe to other operations in that recipe. If one had some 
20 engineering information about these connections one could use this prior infomaation. In the 
absence of engineering priors, one may draw for each production run / the (e — 1) 
connections to operation / uniformly (without replacement) from the set of all operations 
other than i. 

Fixing these e cormections, a preferred embodiment first computes and stores the 
25 realizations of the random variables 4)'(o)'',..., o)^. (where operation / is fixed but the sub- 
recipe (o)'*,..., co^ varies) . It does this for each of the operations i = 1 n. The number of 

vectors needed to compute the landscape f((i)) is thus reduced from s" to n^, (In the case of n 
= 10, J = 3. e = 3, one gets that s" = 3'° > 270 = (10)3^ — n^'.) 

The landscape e(o)) and the method of "walking" on this landscape have now been 
30 completely defined. All that remains to be specified is the starting point (on the landscape) 
for the walk. It should be clear to those of ordinary skill in the art that for some applications. 
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the starting point might be given by information about the production experience of 
competitors or suggestions from the firm*s R&D department. In the absence of such prior 
information, one can merely pick one recipe randomly (with uniform probabilities over the s" 
recipes in the set Q) to be the starting point. 
5 In a preferred embodiment, the log (unless otherwise indicated, log denotes the natural 

logarithm) of the labor requirement is re-normalized so that the adjusted log of the im'tial unit 
labor requirement is 1.0. The relationship between the labor requirement f and the adjusted 
log labor requirement is given by: 

adjusted log labor requirement - I + log t 
10 The adjusted log labor requirement is negative for P < 0.36787. Negative values of this 
convenient measure cause no problems (though it would be economic nonsense \f unadjusted 
9 were to be negative). 

For each realization of a single production run, the estimated progress ratio p is given 

by 

p = 2* 

1 5 where b is the OLS estimate of the learning parameter by i.e., b is the absolute value of the 
slope of the regression line. If t = 1 , then the labor requirement at /, 5, , and cumulative 
output at Yfy correspond to what has been used for estimating b and p in the known art on 
experience curves. If t =^ 1, then will not be equal to the cumulative number of trials that 
define time in the model. The relationship between t (cumulative number of trials). Y, 

20 (cumulative output), and t (number of trials per measured batch) is 




In general Y^ is the appropriate measure of "economic time*' for a given production nm. If 
25 T= 1 , "calendar time** / and "economic time** are the same. Otherwise, / must be adjusted to 
measure economic time. In computational terms, if Ms fixed, then increasing t decreases the 
number of simulated points. Trials take place and labor requirements are modified at each 
date r, but not all modifications are recorded. For example, with t = 20 (and B = 1 as 
assumed), calculation of the per-unit labor requirwnent would not occur until after the 20lh 
30 unit was produced, then again not until the 40th unit, and so on. Under the assumption of / = 
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1000 and t = 1, 1,000 data points in a given simulated experience curve were observed, but 
with T = 20 only 50 data points would be observed. 

Another quantity that can be used in measuring cumulative increases in productivity is 
Hj^ the labor requirement after t trials (or the "terminal" labor requirement, for short). The 
5 focus is on the path of productivity increases, not on the initial labor requirement, so one 
normalizes Jo 1 . The temiinal labor requirement 9f is usually (but not always!) inversely 
related to d : usually, the lower the final labor requirement the steeper is the experience 
curve. If there were no specification error of the experience curve, this would always be the 
case and Hj would be an uninteresting statistic. Consider, however, the case in which there 
1 0 are huge productivity increases in the first few periods after which the labor requirements 
asymptote. For T large, b would then be small because OLS would heavily weight the 
asymptote in this case. Here the fact that >s small indicates that the estimated learning 
coefBcient d (and hence the estimated progress ratio P) might be misleading. 

In order to capture the extent of platcauing, one computes an (inverse) statistic z, 
1 5 improvements percentage per measured batchy defined by 

z= T X number of observed i mprovements 

r 

The measure z is preferable to direct measures of plateauing (e.g., average plateau length) 
20 because it is less sensitive to the distortions caused by the presence of a long final plateau. 
For experience cmve analysis, the transient is of greater interest than the steady state, since in 
most real world cases, the rate of product replacement (due to, say, a new, superior product) 
is r^id relative to the exhaustion of productivity improvements for the original product. The 
total number of improvements observed by t/T is weighted so that the measure will reflect, 
25 not the absolute number of improvements found, but rather the likelihood that a new 
observation will be a productivity improvement. 

Finally, in order to measure the extent of curvature mis-specification in the experience 
curve data, one estimates a second quadratic specification of the learning model: 
log Cy= + tjlog K+ Cj log + c, 
30 where is the labor requirement after cumulative production of Y units. The magnitude, sign 
and level of significance for 4t > the estimate of give an indication of the extent of 
curvature mis-specification in the standard log-log model. A negative and significant 4 
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would suggest thai the log-log form overstates the rate of early, relative to later, productivity 
improvements. This is not a test of the faW SFS effect A negative value of 4 suggests the 
observed run exhibits SF (first 5I0W and then /ast). Often the second S of SFS is outside the 
observed or calculated production run. 
5 FIG. 20 represents a single learning curve for the "base" parameter vector: n = 1 00, 

^=10, e = 5, T-l,6 = l,7= 1000. These parameters were chosen to be reasonable for the 
described simulation. The priors about the validity of these parameters were not very strong. 
Hence sampling was done widely in the space of parameters, but in many of the experiments 
(1 73 experiments out of 423) the parameter space was sampled by moving only one 

10 parameter at a time while holding the others at one of its base values. This is a particular type 
of sensitivity analysis. 

In FIG. 20, the landscape is not perfectly smooth since e = 5 > 1. Plateauing is also 
evident throughout the production run. This is confirmed by the small value of r: only 6.2% 
of the trials lead to improvements in productivity. Overall productivity increase is moderate; 

1 5 this is confirmed by the estimated progress ratio of 87.5%. There is also a positive SFS 
(curvature) effect, but it is not strong. In FIG. 21, two changes have been made in the 
parameters to set e = 1 and n = 1,000, so that the landscape is now as smooth as possible 
(because e=\) and a single step improvement is likely to be small (because n is large and e = 
1). With the smooth landscape, plateauing (except for terminal plateauing) is so reduced that 

20 it cannot be detected by eyeballing the figure. This is confirmed by the value of z: 42.3% of 
the trials result in productivity improvements. The estimated progress ratio of 94.6% 
indicates a relatively slow rate of productivity improvement, but p is upward biased because 
of the SFS effect, which seems to be more pronounced. (One must be careful, however, not 
to visually overweight the qsarse plot in log units for the early periods (relative to the later 

25 periods) in evaluating the SFS effect; the later data points are more crowded together than the 
early data points.) If the quadratic model were fitted to the plot in FIG. 21, one would expect 
^ to be significantly negative, since the plot suggests a concave function. (Note that the plot 
displays clearly the SF of SFS, but that the final S is not displayed in the plot, because of the 
tnmcation at log batch number -7.) 

30 In FIG. 22, the externality parameter e has bcOT increased to its maximum relative to 

/I (e = n = 1 00). The landscape is very mgged because of the very large e. Consequently, 
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plateauing is very strong (only 1% of the trials lead to improvements) and overali progress is 
very small (p = 97.8). 

In FIG. 23, e = I so there are no externalities to cause a rough landscape. But n is 
reduced to 10 while s is increased to 100. The number of operations is few, so that a change 
5 in any operation can be expected to have a large impact on the labor requirement. This is 
reflected by very rapid overall productivity improvement, which is confinmed by the value of 
the estimated progress ratio of 60.2%. Plateauing is evident (z is only 4.4%). The SFS effect 
is evident, although the caution against over-weighting the sparse early realizations also 
applies here. 

1 0 In FIGS. 20-23, only one production run was used. Each of FIGS. 24-28 presents 

averaged data of multiple production runs based on a given vector of parameters. FIG. 24 is 
nearly the same as FIG. 20 except that in FIG. 24 the data is averaged over 20 separate 
production runs. The vertical axis in FIG. 24 measures the adjusted log labor requirement of 
the industry average over 20 firms or, alternatively the adjusted log labor requirement of the 

15 fimi average over 20 plants. The most important difference between the outputs in FIG. 24 
and FIG, 20 is the degree of plateauing. The curve in FIG. 24 is far smoother than the curve 
in FIG. 20. Plateauing is evident in FIG. 20, while it is barely discernible in FIG- 24. This 
is confirmed by the z statistics: in the single run case only 6.2% of the trials resulted in 
improvement, while in the 20-run average, 65% of the trials resulted in improvement. The 

20 estimated progress ratio (87.5) for the single firm walk is very close to the estimated progress 
ratio (86.5) for the average walk of the 20 firms. The small, but positive curvature effect 
seems to be the same for FIGS. 20 and 24. 

FIG. 21 is based on a correlated landscape (e = 1) and hence the single run is quite 
smooth, but some plateauing is discernible. The average walk shown in FIG. 25 is even 

25 smoother. For the single run dqjicted in FIG. 21, p = 94.6% and z = 42.3%. For the 
averaged run depicted in FIG. 25, p = 94.6% and z = 99.9%. The strong positive curvature 
effects in the two figures are nearly identical. 

The landscape behind FIG. 22 is very rugged (e = 100). Plateauing is dominant. 
Only 1% of the trials result in improvements. FIG. 26 is the 20-firm average run using FIG. 

30 22 parameters. Plateauing is reduced; 7.8% of the trials result in improvement. FIG. 27 is 
the 50-fiim average run based on the same rugged landscape data. Plateauing is further 
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reduced; IS. 7% of the trials result in improvements. The estimated progress ratio for each of 
the 3 cases (FIGS. 22, 26, 27) is about 98%. It is difficult to judge the SFS effect when there 
is so much plateauing, but it is positive and one could argue that the effect Is constant across 
FIGS. 22, 26, and 27. 

5 FIG. 28 is the 20-fiTm average related to the single firm walk in FIG. 23. The 

averaging reduces plateauing (increases z), has no effect on the progress ratio or the terminal 
labor requirement, and seems to reduce the SFS effect but only slightly. 

From these experiments, the following conclusions may be reached: (a) averaging 
profoundly reduces plateauing (and increases z); (b) averaging does not substantially affect 
1 0 the estimated progress ratio; and (c) averaging does not seem to have a strong effect on 
curvature or SFS. These are consistent with actual observations of the firm and industry 
experience. 

Many of the qualitative features of experience curves — ranges of values ofp, presence 
of plateaus, different learning rates for the same or similar goods — can be discerned by exam- 
15 ining single realizations of learning curves. However, to study the full effects of changes in 
the values of the underlying parameters on the predictions of the model, one needs to com- 
pute more than one realization per set of parameter values. Consequently, for each chosen 
parameter vector in the experiment, 20 independent realizations were computed 

In the simulations, computing 20 different realizations means running the simulation 
20 program using the same parameter set, but with 20 different random seeds. A new random 
seed yields a new realization of the externality connection among the operations, a new 
realization of the laiidscape I(q), a new starting point on the landscape and hence a new 
experience curve. Hence in this class of models the "maximum degree of randomness" 
betweai different realizations of the experience curve was chosen* (Even if the sets of 
25 connections £, (/ - 1 ,...,n), the realization of the landscape, {(w), and the starting recipe, Wq, 
were all to be held constant, different realizations of the experience curve would still be 
possible (indeed almost certain) due to different sequences of recipe sampling.) The sample 
standard deviations computed are therefore likely to be biased upward. 

The set of potentially interesting parameters is large. Computations were restricted to 
30 the following grid-like parameter space: 

rt = l,lO, 20, 50. 100, 500,1000; 
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5 « 2, 10,25.50.75,100; 

e «= 1. 2, 3, 5. 6, 7, 8. 9,10, 25, 50, 75, 100 (but with the constraint that e s n); 
5 = 1. 2, 4,10, 25, 50, 75, 100 (but with the constraint that 6 s n); 
T = 1, 10. 50, 100. 250 (but with the constraint that x s 7); 
5 . r= 100.500. 1000.5000. 

Two subsets of parameters were used. The first subset, the focused set of parameter 
values, reflects the priors informed by a review of the empirical literature, introspection about 
production processes, and comparisons with the modeling in evolutionary biology. To 
achieve the rapid productivity increases that have been observed, rather small values of e and 
10 6 relative to n were focused on (in particular e = 1 and 5 and 5 - 1 ), as well as relatively low 
values of s (in particular 5 - 2 and 10). Actually 5 is not a small value for the parameter e 
when /J = 100 or /I - 1,000. In fact, e — 1 gives the number of externalities per operation, so 
the number of externalities per recipe would be n{e — 1). The length of the run was 
frequently set at r= 1000. This was chosen to reduce or eliminate the effects of the terminal 
1 5 plateau. See the center panel in FIG. 35, which confirms our strategy. For s = 10. n = 100, e 
= 5, the mean progress ratio is smallest for r= 1000. For T> 1000 the effect of the terminal 
plateau is to reduce the progress ratio. For many runs, /i = 1 00 was adopted for the number of 
operations. For many runs x = 1 was adopted. The base cases for the focused runs are then: 
(« = 100,5 = 2,e=1.6=l,T = l,r=IOOO) 
20 . (n = lOO,5=10.e=l. 6=1, T = l.r= 1000) 

(n = 100. 5 = 2. e = 5. 6 = I, T = 1, r= 1000) 
{n - 100. ^ - 10, e = 5, 6 = 1, T - 1. r= 1000) 
The focused parameter set was constructed from the above four base case vectors by 
varying the six parameters one at a time. Summary statistics for the focused parameter set are 
25 given in Table 3 (FIG. 8). There are 1 73 parameter vectors in the focused set. For each 
vector there are 20 runs, so the total number of runs is 3,460. 

If the parameter space were only of dimension 2 and there were only a single base 
case, the method of choosing parameters would be that described in FIG. 29. Say the base 
parameter vector is given as row 3. column 5 of the simple 10x10 matrix recipe set in FIG. 
30 29. Then the set of focused parameters is the union of the set of recipes having a row-3 
component with the set of recipes having a coiumn-5 component. The focused parameter set 
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is in the shaded *'cross." The advantage of the focused parameter method is that one begins 
with a set of reasonable base parameters and then tests the sensitivity of the predictions to 
each of these parameters varied one at a time from each of the base vectors. The parameters 
are often set at "extreme" values to lest parameter sensitivity. On the other hand, as can be 
5 seen fipom FIG. 29, this selection of parameter points is clearly "statistically inefficient" and 
results could be sensitive to a choice of base parameter vectors. As a counter to these 
potential biases, experience curves based on a random set of parameter values were also 
simulated. The random parameter vectors were chosen as follows: n drawn uniformly from 
{l,...,1000}; J drawn uniformly from { 1,...,1 00} ; e drawn uniformly from {!,.„, 
10 min(lO^)}; <5drawn uniformly from {I,. ..,min(10,n/2)} ; rdrawn uniformly from {l....,IO}; 
and r= 1000. 

The number of parameter vectors in the set of random parameters is 250. There are 
20 nans per vector. Hence there are 5000 runs in all for the random parameter set. Summary 
statistics for the random parameter set are given in Table 4 (FIG. 9). 

1 5 The parameters selected for the focused set suffer from "FIG. 29 bias"; the 

parameters selected for the random set do not These are not the only differences between the 
two parameter sets. See Tables 3 and 4. For the parameters n and 5, the ranges and the 
means are greater in the random set than in the focused set, but this is reversed for e and 6. In 
the focused set, t was varied between 10 and 1000, but in the "random" set Twas fixed at 

20 1000. The means of the predicted ps are similar, but the range and the standard deviations of 
theps are much larger for the focused set. The mean of the Js, the range, and the standard 
deviation are larger for the focused set Curvature predictions and labor requirements were 
not assembled for the focused set 

The effects of varying the parameters of the model were evaluated using two scoring 

25 methods. For the focused parameter set. judgment from studying all the relevant simulations 
was used- This is called here *the eyeball method of scoring" paramtter effects. The eyeball 
method is very illuminating, but it does require judgment from the modeler. As a guard 
against human bias, simple OLS regression scoring for the focused parameters (173 
experiments) and the random parameters (250 experiments) was also used. The basic 

30 disadvantage to OLS scoring is that the results are "global": "extreme" parameter values are 
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probably overweighted. Furthermore, subtle non-raonotonic interrelationships are obscured 
by the simple functional forms used for OLS scoring. 

The effect of varying a particular parameter on one of the predictions (or **results**) of 
the model typically depends on the interplay of two effects: (1) the effect of the parameter 
5 change on the size of the recipe space and (2) the effect of the change on the trial (or, recipe 
sampling) mechanics. Typically, these effects arc too complicated to permit an analytic 
solution, especially since the bias is on the short term and the medium term. Computation is 
called for. 

The first effect of increasing n is to increase the number of recipes, s". This effect 

10 (especially for large T) should tend to increase long-term productivity improvement On the 
other hand, increasing n decreases (especially when e is small) the expected cost reduction of 
one-step or few-step changes in the recipe on unit labor costs, because — given the assumption 
of additive unit costs — with more operations each operation contributes less to the overall 
cost. This suggests that increasing n might decrease the rate of short-term (and medium- 

15 term) productivity improvement. 

In FIG. 30, the sample means, /T, (over 20 nms) and the sample standard deviations, 
are plotted. It was observed that for small n the effect of increasing n is to decrease ^(i.e., to 
increase the mean rate of productivity improvement), but for larger n, the effect on p of 
increasing n is positive. As n becomes even larger, the effect on p of increasing n attenuates. 

20 The standard deviation, j^, is decreasing in n. 

The sole effect of increasing the parameter s on the rate of productivity improvement 
is through increasing the size of the recipe space and hence through increasing long-run 
productivity improvement. From FIG. 31, it is apparent that this is confirmed. Increasing s 
substantially decreases ^ while slightly increasing s^^ but each effect eventually atteniiates as s 

25 becomes very large. In particular, for s larger than 50 the effects on ^of increasing s are 
negligible. 

The most obvious effect of increasing e is increasing the ruggedness of the landscape, 
thereby reducing the effectiveness of the myopic recipe sampling procedure. As e increases 
the number of local optima in the landsc^e increases, thus the probability of being 'Vapped** 
30 increases. This reasoning suggests thatp would tend to be increasing in e. On the other 
hand, increasing the parameter e increases the number of cosi-relevant sub-recipes (equal to 
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ns^, and thus reduces the expected value of the global minimum labor requirement. 
Furthermore, increasing e has the effect of speeding the rate of experimentation, since each 
trial modifies the contribution to the labor requirement of not one, but e different operations 
within the production recipe. For these last two (related) reasons, one might expect ^to tend 
5 to be decreasing in 

Results for e are given in FIG. 32 and HG. 33. If the recipe space is relatively small 
(j < 6, rt = 100), then increasing e seems to decrease p for e small, although the standard 
errors suggest that one should be cautious in making this conclusion. The smallest^ (and the 
lowest terminal labor requirements) seems to occur at values of e * 5. If the recipe space is 
10 larger (5 > 6, /I = 100), thenp and are clearly monotonically increasing in e. For j large, the 
transitory effects ofe depend on how much progress has already been made. If C(u,) is above 
the expected value of (t)((o) over all of Q, then increasing e (for small values of <?) increases 
the expected rate of short-run productivity improvement. On the other hand, if P((i>J is below 
the expected value of <J>(<o), then increasing e (for small values ofe), decreases the expected 
15 rale of productivity improvement. See Auerswald ( 1 998, Chapter 3) (P. Auerswald, 
Organizational Learning, Production Intranalities and Industry Evolution, University of 
Washington Ph.D. thesis, 1998). This is because the ruggedness is helpful in "bad" (high 0 
neighborhoods, but hurtful in good (low 0 neighborfioods. This phenomenon is seen in the 
simulations. This argument also suggests that increasing e tends to increase the curvature 
20 mis-specification. This effect is present in the simulations. 

Taking bigger steps on a given landscape is somewhat like walking with smaller steps 
on a more rugged landscape. Hence, increasing 6 should be analogous to increasing e. Like 
e, increasing 6 increases except for some cases with 6 small; see FIG. 34. There are 
suggestions in the data that for small 6 and appropriately chosen values of the other 
25 parameters,^ is decreasing in 6. Indeed, the parameters e and 6 are close cousins. (They 
would be even closer if by increasing 5 one would be increasing the size of every "step*' 
rather than merely increasing the maximum step size.) 

The length of the production run would not affect p if the model were perfectly 
specified, in particular, if a power law fitted the data well. Varying Tprovides a method for 
30 analyzing the cuivatore mis-specification of the experience curve. See FIG. 35. For the case 
with a relatively small recipe space {s = 10, n = 100) and a relatively rugged landscape (e = 
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5), the SFS effect is pronounced. The progress ratio falls from T~ 1 to about T= 1000 and 
then gradually rises out to 7* = 5000. This pattern is also suggested for the case: 5 « 100, /i = 
10, e = 5, but the standard enors are too lai:ge for confidence. For the case: s-\0,n- 1000, 
e = 1, the recipe space is very large and the landscape is smooth. The estimated mean 
5 progress ratio is monotonically decreasing (with small standard errors) over I to T= 
5000. The recipe space is so large in this case that the effects of the second 51ow response do 
not kick in sufficiently to increase p even at 7"== 5000. 

The statistic z (percent of trials that resuh in improved productivity) is an inverse 
measure of plateauing. Increasing the number of recipes by increasing n decreases plateauing 

10 with no discernible effect on the standard deviation s, (see FIG. 37). hicreasing the number 
of recipes by increasing 5 decreases plateauing but increases (except for large values ofs) the 
standard deviation Increasing e, in general, increases plateauing (See FIG. 38). Increasing 
6, in general, increases plateauing, but for small <?, j, and 6» it appears that increasing 6 
reduces plateauing (see FIG. 39). 

15 The positive curvature effect is pronounced in FIGS, 21 & 25 (smooth landscape 

caseswith: n = 1000,.?= 10, e= 1. 6 = 1, r= 1, r= 1000). Linear OLS in log^og space at 
first overstates the actual rate of productivity improvement and then understates it. 
Ultimately (because of the terminal plateau), linear OLS in log/log space can be expected to 
overstate the rate of productivity improvement. This is not surprising for smooth, many- 

20 recipe landscapes with rt large and 6 smail. Because n is large, for each improvement the 
expected reduction in the labor requirement is small. Because the landscape is smooth, the 
probability of finding an improvement on any given trial is relatively high. The large recipe 
set implies that the stock of potential improvements is being exhausted at a very slow rate. 
Hence the resulting productivity plot is likely to be nearly linear in natural units. Therefore it 

25 will be strongly concave to the origin in log/log space. This is in agreement with the results 
of the model and with observed experience curves. The data behind experience curves 
fluently show curvature bias, usually positive (indicating a concave function), but 
sometimes negative (indicating a convex function). In the discussion above on the effects of 
7 on and separate simulation evidence on curvature and SFS are also provided. 

30 For the two parameter sets (focused and random), each of the predictions of the model 

was regressed on the parameters of the model. More efficient grids and pooling of data ar« 
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known to those of ordinary skill in the art. See» e.g., Judd (1998, Chapter 9) QL Judd, 
Numerical Methods in Economics and Finance, 1998) on acceleration techniques for Monte 
Carlo experiments. Various functional forms were employed. The results arc not very 
sensitive to the functional form. Reported here are the regression results for cases in which 
5 the prediction in natural units is regressed on log values of the parameters. 

In Table 5 (FIG. 10) and Table 6 (FIG. 11) are summarized the results of OLS 
scoring for prediction of ^. The R^s are not very high. This is probably because the 
functional form being used is highly mis-specified. Eyeball scoring indicated that the 
interaction effects of the parameters can be quite subtle and tfiat the effects of varying even a 

10 single parameter are not monotone. Nonetheless, the / tests yield high levels of significance 
for most of the parameters. The R^s and the / tests are more favorable for the random set. 
This is probably because the random set does not suffer from "FIG. 29 bias," i.e.. (1) the data 
is more dispersed for the random set and (2) the focused set conditions more on the 
interesting small values for which the results tend to be non-monotone in the parameters. 

15 Both OLS scorings predict that increasing either of the two cousin parameters — e and 6 — will 
increase p. From the focused parameter set, the prediction is that increasing s decreases p. 
The prediction f^om the random parameter set is the same but at a lower level of significance. 
From the random set, there is a weak prediction that increasing t increases p from the focused 
set, the coefBcient is not significant. This weak effect of t on p is probably because of mis- 

20 specification of the experience curve: as t is increased initial productivity improvements can 
be so rapid that the terminal plateau is reached quickly The estimated progress ratio then 
gives a downward biased estimate of the "actual rate of productivity improvement." Another 
reason that the t effects might appear to be weaker than expected is that the model of 
reporting and implementation of quality control trials might not be the best one. For the 

25 focused parameter set, increasing the parameter n decreases the prediction p. For the random 
parameter set, increasing n strongly increases p. This can be explained by the fact that the 
random parameter set contains bigger /is than does the focused parameter set; see Tables 3 
and 4. Eyeball scoring strongly suggests that pis not monotone in n. For smaller n,p\s 
decreasing m w. For larger «, p is increasing in n. 

30 Tables 7-8 (FIGS. 12-13) report the prediction of the standard deviation s^,. 

Increasing n reduces the predicted s^. Increasing s increases the predicted from focused 
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parameters. Increasing e decreases the predictions of j^. The predicted effects of varying 6 
differ between the two parameter sets. Increasing -c decreases the prediction of j^. 

See Table 9 (FIG. 14). Increasing n increases the predicted mean improvement 
percentage, F while increasing e or 6 decreases z. 
5 If Cj is negative then the estimated quadratic experience curve is a concave function in 

log/log space. Sec Table 10. Increasing n, or t decreases predicted and hence increases 
the curvature effect. Increasing ^ or 6 increases the predicted and hence decreases the 
curvature effect 

The observed values of (from actual firms and industries) are likely to be biased 
10 downward, since estimates of the progress ratio also pick up the effects of increasing returns 
to scale in production and of the development (the D of R&D) activity devoted to 
improvements in production efficiencies. 

It is straightforward to test the predictions of a model in the following way: (1) take 
some learning curves for a particular industry; (2) adjust the reported statistics (such as ^ and 
1 5 for the numbers of observations in each run; then (3) find in the set of predictions the 
parameter sets for which the predictions are near the observed statistics. 

FIG. 40 discloses a representative computer system 401 0 in conjunction with which 
the embodiments of the present invention may be implemented. Computer system 4010 may 
be a personal computer, workstation, or a larger system such as a minicomputer. However, 
20 one skilled in the art of computer systems will understand that the present invention is not 
limited to a particular class or model of computer. 

As shown in FIG. 40,representative computer system 4010 includes a central 
processing unit (CPU) 4012, a memory unit 4014, one or more storage devices 4016, an input 
device 4018, an output device 4020, and communication interface 4022. A Systran bus 4024 
25 is provided for communications between these elements. Computer system 4010 may 
additionally function through use of an operating system such as Windows, DOS, or UNIX. 
However, one skilled in the art of computer systems will understand that the present 
mvention is not limited to a particular configuration or operating system. 

Storage devices 4016 may illustratively include one or more floppy or hard disk 
30 drives. CD-ROMs, DVDs, or tapes. Input device 4018 comprises a keyboard, mouse, 

microphone, or other similar device. Output device 4020 is a computer monitor or any other 
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known computer output device. Communication interface 4022 may be a modem, a network 
interface, or other connection to external electronic devices, such as a serial or parallel port 

While the subject invention has been particularly described with reference to a 
preferred embodiment, it will be appreciated by those of ordinary skill in the art that various 
5 changes and modifications may be made without departing from the spirit and scope of the 
method. For example, the leaming-by-doing model is sufiQciently rich to match the reported 
progress ratios fix)m estimated experience curves. Those of ordinary skill in the art will 
recognize that certain modifications arc routine. For example, if one has data or priors on the 
values of p, ^, z, , Cj , s^^ , or any of the other predictions for a particular plant, firm or 

10 industry producing a specific good, then one can search for parameters n, 5, 5 and x that 
predict the data. One might also have priors on some of the parameters based on engineering 
considerations or (especially in the case of 7) market considerations. From this, one could 
come up with a best explanation of the observed data. How well the modified model fit 
observations and priors would measure the usefulness of the modified model of leaming-by- 

15 doing. 

It is intended that the appended claims be interpreted to cover the foregoing as well as 
equivalents thereto. 



37 



Claims 



5 



10 



15 



20 



25 



30 



35 



40 



45 



55 



wo 00/20983 



PCrAJS99/229n 



CLAIMS 

What is claimed iS: 

1 . A metiiod for determining a production plan comprising the steps of: 

5 defimng a plurality of production recipes such that each of said production 

recipes is a vector of n operations; 

selecting a current one of said production recipes; 
evaluating said current production recipe to determine its cost; 
modifying said current production recipe to create a trial production recipe; 
to evaluating said trial production recipe to determine its cost; and 

assigning said trial production recipe to said current production recipe if said 
cost of said trial production recipe is less than said cost of said current production recipe. 

2. A method for determining a production plan as in claim 1 further comprising 
15 the steps of repeating said modifying said current production recipe step, said evaluating said 

trial production recipe step and said assigning said trial production recipe step. 

3. A method for determining a production plan as in claim 1 wherein each of said 
operations of said production recipes has s possible settings. 

20 

4. A method for determining a production plan as in claim I wherein said cost of 
said production recipe is a labor cost. 

5. A method for determining a production plan as in claim 3 further comprising 
25 the step of defimng a distance between said current production recipe and said trial 

production recipe. 

6. A method for determining a production plan as in claim 5 further comprising 
the step of defining a distance between a first one of said operations and a second one of said 

30 operations. 
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7. A method for determining a production plan as in claim 6 wherein said 
distance between said first one of said operations and said second one of said operations is 
defined as one if said first operation is not equal to said second operation and as zero 
otherwise. 

5 

8. A method for determining a production plan as in claim 6 wherein said 
distance between said first one of said operations and said second one of said operations is 
defined as the difference between said setting of said first operation and said setting of said 
second operation. 

10 

9. A method for determining a production plan as in claim 6 wherein said 
distance between said current production recipe and said trial production recipe is defined as 
a sum of said distance between said operations of said current production recipe and 
corresponding ones of said operations of said trial production recipe. 

15 

1 0. A method for determining a production plan as in claim 9 wherein said 
modifying step creates said trial production recipe in a neighborhood of said current 
production recipe. 

20 11. A method for determining a production plan as in claim 1 0 furth^ comprising 

the step of defining said neighborhood of said current production recipe as those of said 
plurality of production recipes that arc within a maximum distance from said current 
production recipe. 

25 12. A method for determining a production plan as in claim 1 further comprising 

the step of defining a connectivity indicator e/ as 1 if said setting of said operation i affects 
said cost of said operation j and 0 otherwise. 

13. A method for determining a production plan as in claim 1 2 further comprising 
30 the step of defining a number e^ of operations that affect said cost of operation i as a sura of 
said connectivity indicators Cj* over j fix)m 1 to n. 
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14. A method for detenninmg a production plan as in claim 1 3 further comprising 
the step of defining said number as an externality parameter e. 

15. A method for determining a production plan as in claim 1 wherein said 
5 modifying step is a non-conunitted experiment to said trial production recipe. 

16. A method for detemiining a production plan as in claim 1 wherein said 
modifying step is a commitment to said trial production. 

10 17. A method for determining a production plan as in claim 1 wherein said 

plurality of production recipes comprise backward technologies. 

18. A method for determining a production plan as in claim I wherein said 
plurality of production recipes comprise advanced technologies. 

15 

1 9. A method for determining a production plan as in claim 1 wherein said 
plurality of production recipes comprise nascent technologies. 

20. A method for predicting technological irmovation comprising the steps of: 
20 defining a model comprising: 

a plurality of production recipes such that each of said production 
recipes is a vector of n operations; and 
a plurality of model parameters; and 
executing said model comprising the steps of: 
25 selecting a current production recipe; 

evaluating said current production recipe to determine its cost; 
modifying said current production recipe to create a trail production 

recipe; and 

assigning said trial production recipe to said current production recipe 
30 if said cost of said trial production recipe is less than said cost of said current production 
recipe. 
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21 A method for predicting technologica] innovation as in claim 20 wherein said 
executing said model step further comprises the steps of: 

repeating said modifying said current production recipe step, said evaluating 
said trial production recipe step and said assigning said trail production recipe step. 

5 

22. A method for predicting technological innovation as in claim 20 wherem said 
plurality of model parameters comprise a possible settings parameter s defining a number of 
possible settings of said operations of said production recipes. 

10 23. A method for predicting technological innovation as in claim 20 wherein said 

plurality of model param^ers comprise a connectivity indicator e^' defined as 1 if said setting 
of said operation i affects said cost of said operation j and 0 otherwise. 

24. A method for predicting technological innovation as in claim 23 wherein said 
15 plurality of model parameters further comprise a number of operations that affect said cost 

of operation i as a sum of said connectivity indicators e/ over j firom 1 to /i. 

25. A method for predicting technological irmovalion as in claim 24 wherein 
number Cj is defined to be a predetermined externality parameter e. 

20 

26. A method for predicting technological irmovalion as in claim 20 wherein said 
plurality of model parameters comprise a recipe distance between said current production 
recipe and said trial production recipe. 

25 27. A method for predicting technological innovation as in claim 26 wherein said 

plurality of model parameters further comprise an operation distance between a first one of 
said operations and a second one of said operations. 

28. A method for predicting technological irmovalion as in claim 27 whmin said 
30 operation distance between said first one of said operations and said second one of said 
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Operations is defined as one if said first operation is not equal to said second operation and as 
zero otherwise. 

29. A method for predicting technological innovation as in claim 27 >yherein said 
5 operation distance between said first one of said operations and said second one of said 

operations is defined as the difference between said setting of said first operation and said 
setting of said second operation. 

30. A method for predicting technological innovation as in claim 27 wherein said 
10 recipe distance between said current production recipe and said trial production recipe is 

defined as a sum of said operation distances between said operations of said current 
production recipe and corresponding ones of said operations of said trial production recipe. 

31. A method for predicting technological innovation as in claim 26 wherein said 
15 plurality of model parameters further comprise a maximum distance from said current 

production recipe, said maximum distance defining a neighborhood of said current 
production recipe. 

32. A method for predicting technological innovation as in claim 3 1 wherein said 
20 modifying said current production recipe step comprises the step of creating a trial production 

recipe within said neighborhood of said current production recipe. 

33. A method for predicting technological innovation as in claim 20 wherein said 
plurality of model parameters further comprise a production run length. 

25 

34. A method for predicting technological innovation as in claim 33 wherein said 
executing said model step is performed for said production run length. 

35. A method for predicting technological innovation as in claim 20 further 
30 comprising the step of defining values for said plurality of model parameters. 
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36. A method for predicting technological innovation as in claim 35 further 
comprising the step of defining at least one innovation prediction variable. 

37. A method for predicting technological innovation as in claim 36 wherein said 
5 executing said model step detennines values of said at least one innovation prediction 

variable corresponding to said values of said model parameters. 

38. A method for predicting technological innovation as in claim 36 wherein said 
at least one innovation prediction variable comprises a least one estimated progress ratio. 

10 

39. A method for predicting technological innovation as in claim 38 wherein said 
at least one estimated progress ratio is defined as 

P = 2-'. 

wherein bis the OLS estimate of the learning coefficient 6. wherein the learning coefficient b 
represents the rate at which productivity increases as a firm acquires experience, and wherein 
15 (a) ^ is the absolute value of the slope of the regression line in log^og 

space given by 

log ?, = log<i-^i logy,.,. 

where - f(<i),) is a unit labor requirement for production run r, and wherein a is the labor 
20 needed to produce a first batch of a good; 

(b) y,. , is defmed by y^^, = ^ y^, a cumulative output up to (but not 

including) production run t; 

(c) T = — , wherein a trials parameter t satisfies 

25 
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X - B/B , wherein B is a measured batch size and B is a quality control 

batch size; 

(d) P(<*>i) is defined to be the realization of a random field 4>(w), wherein 
<|)((i)) is defined by 

5 

<t>(w) = i (t)'((i>). 



wherein for each /, 4»'{w) is the unit labor cost of operation /. 

10 

40. A method for predicting technological innovation as in claim 38 wherein said 
at least one innovation prediction variable further comprises a sample standard deviation of 
said at least one estimated progress ratio. 

15 41 . A method for predicting technological innovation as in claim 35 further 

comprising the step of defining at least one measure of model misspecification. 

42. A method for predicting technological innovation as in claim 41 wherein said 
at least one measure of model misspecification comprises a plateau effect 

20 

43. A method for predicting technological innovation as in claim 42 wherein said 
at least one measure of model misspecification comprises a curvature misspecification. 

44. A method for predicting technological innovation as in claim 41 wherein said 
25 executing said model step determines values of said at least one measure of model 

misspecification corresponding to said values of said model parameters. 

45. Computer executable software code stored on a computer readable medium, 
the code for determining a production plan, the code comprising: 

30 code to define a plurality of production recipes such that each of said 

production recipes is a vector of n operations; 
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code to select a current one of said pTx>duction recipes; 

code to evaluate said current production recipe to detennine its cost; 

code to modify said current production recipe to create a trial production 

recipe; 

5 code to evaluate said trial production recipe to detennine its cost; and 

code to assign said trial production recipe to said current production recipe if 
said cost of said trial production recipe is less than said cost of said current production recipe. 

46. A programmed computer system for determining a production plan comprising 
1 0 at least one memory having at least one region storing computer executable program code and 

at least one processor for executing the program code stored in said memory, wherein the 
program code includes: 

code to define a plurality of production recipes such that each of said 
production recipes is a vector of n operations; 
1 5 code to select a current one of said production recipes; 

code to evaluate said cunrent production recipe to determine its cost; 

code to modify said current production recipe to create a trial production 

recipe; 

code to evaluate said trial production recipe to determine its cost; and 
20 code to assign said trial production recipe to said current production recipe if 

said cost of said trial production recipe is less than said cost of said current production recipe. 

47. Computer executable software code stored on a computer readable medium, 
the code for predicting technological innovation, the code comprising: 

25 code to define a model comprising: 

a plurality of production recipes such that each of said production 
recipes is a vector of n operations; and 

a plurality of model parameters; and 
code to execute said model comprising: 
30 code to select a current production recipe; 

code to evaluate said current production recipe to determine its cost; 
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code to modify said current production recipe to create a trail 

production recipe; and 

code lo assign said trial production recipe to said current production 
iO recipe if said cost of said trial production recipe is less than said cost of said current 

5 production recipe. 



15 



48. A programmed computer system for predicting technological innovation 
comprising at least one memory having at least one region storing computer executable 
program code and at least one processor for executing the program code stored in said 
10 memory, wherein the program code includes: 
20 code to defme a model comprising: 

a plurality of production recipes such that each of said production 
recipes is a vector of ;i operations; and 

a plurality of model parameters; and 

25 

15 code to execute said model comprismg: 

code to select a current production recipe; 

code to evaluate said current production recipe to determine its cost; 
30 code to modify said current production recipe to create a trail 

production recipe; and 

20 code to assign said trial production recipe to said current production 

recipe if said cost of said trial production recipe is less than said cost of said current 
production recipe. 
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